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DETAILED ACTION 
Information Disclosure Statement 

The information disclosure statement (IDS) submitted on 2/10/2004 has been considered 
by the examiner. 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of ah application filed in the United States only if the international application designated the United 
States and was published under Article 2 1 (2) of such treaty in the English language. 

Claims 1-15, 17, 19-37, 39, 41-44 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Watanabe et al. U.S. Patent No. 6,763,284 (hereinafter Watanabe). 

Re Claims 1 and 23: 

Watanabe discloses method for fading computer-generated information into an image of 
the real environment detected by an image receiving unit located on a viewing device, 

Wherein there is a determination of a position and an orientation or pose of the image 
receiving unit (e.g., the 2-dimensional image pickup device 2 or 12 in Fig. 1 are mounted on the 
robot and the reference work 7 in the field of view is captured using the camera 12 and the 
image is display on the image display device 3. A position to be measured by the first robot 5 is 
pointed on an image using a pointing device 4 and coordinates of the position pointed on the 
image are stored in the robot controller and thereby obtaining/determining the coordinates of 
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the position. A view line corresponding to the position is obtained on the 3-dimensional image 
and the position and the orientation of the camera 12; column 11-12. Moreover, the 3- 
dimensional positions of a sequence of the points which constitute the working line thus obtained 
are utilized as teaching points of the robot and thus constitute the robot specific information as 
teaching points of the robot to produce the movement path or a motion program of the robot 5, 
see column 7, lines 20-35. It is also disclosed that the orientations of faces in the vicinity of the 
working line are measured in correspondence to the respective points: The measurement start 
point is selected on a working line on which an actual working is performed. The robot having 
the image pickup device 2 or 12 constitutes an image receiving unit) and that robot-specific 
information (e.g., the working line or the working path of the robot is the robot-specific 
information; column 8, lines 50-60) corresponding to this determination is faded over the image 
of the real environment on the viewing device (e.g., the image of the camera 2 or 12 is displayed 
on a screen of the image display device 3 and the detected point positions constituting the 
working line are displayed while being superimposed on this image; column 5, lines 10-17 and 
column 12, lines 45-67). 
Claims 2 and 24: 

Watanabe further discloses a plurality of coordinate systems including the world 
coordinate system and the coordinates of the positions of the working path are detected and 
superimposed on the image of the reference work (e.g., column 12, lines 10-21). 

Claims 3 and 25: 
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Watanabe further discloses a plurality of coordinates being faded/superimposed on the 
reference image and the coordinates are given as coordinates to a tool coordinate system fixed on 
the distal end portion of the hand of the robot (column 2, lines 40-49). 

Claims 4 and 25: 

Watanabe further discloses the coordinate positions of the working path being 
faded/superimposed on the reference image or the working object relative to the axes of the 
three-dimensional or two-dimensional space (column 2, lines 28-49). 

Claims 5 and 27: 

Watanabe further discloses an image of a control element (working path) of a robot 
manual programmer (operator) movable in at least two dimensions (either two-dimensional 
space or three-dimensional space) in faded in /superimposed over the image of the working 
object {column 5, lines 10-17 and column 12, lines 45-67). 

Claims 6 and 28: 

Watanabe further discloses an image of a control element such as a working path of a 
robot and the orientation of the robot hand are utilized (column 4 and column 7, lines 20-40). 
Claims 7 and 29: 

Watanabe further discloses at least one tool moved by a robot (Figs. 3-5), preferably 
several robot elements are faded into a working environment of a robot (Figs. 3-5). 
Re Claims 8 and 30: 

Watanabe further discloses teaching an attitude of a tool center point to the robot and a 
working path in relation to the reference work (column 7-8) and the thinning process of 
redundant detected points (column 7-8). 
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Re Claims 9 and 31: 

Watanabe further discloses an image of a control element (working path) of a robot 
manual programmer (operator) movable in at least two dimensions (either two-dimensional 
space or three-dimensional space) in faded in /superimposed over the image of the working 
object (column 5, lines 10-17 and column 12, lines 45-67). 

Claims 10 and 32: 

Watanabe further discloses the coordinate positions of the working path being 
faded/superimposed on the reference image or the working object relative to the axes of the 
three-dimensional or two-dimensional space (column 2, lines 28-49). 

Claims 11 and 33: 

Watanabe further discloses adapting a robot working path to the position of a detected, 
real work-piece or the working object, a virtual image of the working object with a robot path 
adapted thereto is faded in, so that by superimposing the virtual work object image with the 
image of the real object it is possible to adapt the robot path to be performed to the position of 
the real working object (column 7-8 and column 12). 

Claims 12 and 34: 

Watanabe further discloses the working area reachable by a robot and/or a permitted 
operating area is visualized on the viewing device (column 12). 
Claims 13 and 35: 
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Watanabe further discloses movement corridors of a robot tool, robot hand and/or further 
robot elements are visualized on the viewing device (column 5, lines 10-30 and column 12, lines 
45-67). 

Claims 14 and 36: 

Watanabe further discloses permanent and/or instantaneous associations of at least one 
manual programmer of at least one robot are visualized (column 12, lines 45-67). 
Claims 15 and 37: 

Watanabe further discloses the position and orientation of the display are detected by 
fixed markings in space (column 12, lines 45-67). 
Claims 17 and 38: 

; Watanabe further discloses the position and orientation of the viewing device are 
determined optically (column 12, lines 45-67). 
Claims 19 and 39: 

Watanabe further discloses the robot-specific information (e.g., the working line or the 
working path of the robot is the robot-specific information; column 8, lines 50-60) corresponding 
to this determination is faded over the image of the real environment on the viewing device (e.g., 
the image of the camera 2 or 12 is displayed on a screen of the image display device 3 and the 
detected point positions constituting the working line are displayed while being superimposed 
on this image; column 5, lines 10-17 and column 12, lines 45-67 and column 13, lines 1-46). 

Claims 20-22 and 40-44: 

Watanabe further discloses the robot-specific information (e.g., the working line or the 
working path of the robot is the robot-specific information; column 8, lines 50-60) corresponding 
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to this determination is faded over the image of the real environment on the viewing device (e.g., 
the image of the camera 2 or 12 is displayed on a screen of the image display device 3 and the 
detected point positions constituting the working lime are displayed while being superimposed 
on this image; column 5, lines 10-17 and column 12, lines 45-67 and column 13, lines 1-46). 

Claim Rejections -35 USC §103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 16 and 38 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Watanabe et al. U.S. Patent No. 6,763,284 (hereinafter Watanabe). 

Claims 16 and 38: 

The claims further recite markings are detected by radio receivers. Watanabe is silent to 
the claim limitation. 

It would have been obvious to have incorporated radio receivers to collect the working 
path information or to serve as an image pickup device because Watanabe discloses a general 
image pickup device such as a camera or an optical receiver for collecting the path information 
(Watanabe Figs. 1-5) and thereby suggesting the claim limitation of an image pickup device such 
as the radio receivers. Moreover, the radio receiver can be used in replace with the optical 
receiver as an alternative image and information collection device. 
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One of the ordinary skill in the art would have been motivated to use an alternative 
receiver to collect the robot working path information and to pick up the image information 
when necessary (column 9-10). 

Claims 18 and 40 are rejected under 35 U.S. C. 103(a) as being unpatentable over 
Watanabe et al. U.S. Patent No. 6,763,284 (hereinafter Watanabe) in view of Mizuno et al. U.S. 
Patent No. 5,876,325 (hereinafter Mizuno). 

Claims 18 and 40: 

The claims further recite data spectacles to be worn by a user for displaying the robot 
information. Watanabe is silent to the claim limitation. 

However, Mizuno discloses HMD for displaying the robot information (Mizuno Fig. 28 
and 35). 

It would have been obvious to have incorporated HMD to display the robot information 
because Watanabe discloses a display device connected to the robot for collecting the robot 
specific information (Watanabe Figs. 1-5) and Mizuno discloses HMD coupled to the robot 
manipulators for collecting the robot specific information (Mizuno Figs. 28 and 35) and therefore 
an alternative display device can be used to collect the robot specific information. 

One of the ordinary skill in the art would have been motivated to use an alternative 
display device such as an HMD so that an operator armed with HMD can directly see the work 
performed while specifying the points along the working path (Watanabe column 12, lines 45- 
67). 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (571) 272-7665. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (571) 272-7664. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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